Intelligent media synchronization

ABSTRACT

Intelligent synchronization of media or other material output from multiple media devices is contemplated. The intelligence synchronization may include instructing the media devices to coordinate playback in concert with a conductor whereby the conductor acts a focal point or reference for the non-conducting media devices. The non-conductor may transmit sync messaging having data or other information sufficient to facilitate coordinating operation of the non-conductors in a manner sufficient to synchronize output of the media.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. provisional Application No.62/189,702 filed Jul. 7, 2015, the disclosure of which is incorporatedin its entirety by reference herein.

TECHNICAL FIELD

The present invention relates to facilitating intelligent mediasynchronization, such as but not necessarily limited to intelligentlysynchronizing playback of sensory or other human-perceptible media froma plurality of output devices within auditory and/or visual range ofeach other.

BACKGROUND

The playback of media or other human-perceptible content from multipleoutput devices can be problematic when the output devices are withinauditory and/or visual range to each other, or more particularly aperson within range of two or more of the output devices, as the personmay perceive auditory and/or visual differences depending on how theoutput devices play or otherwise deliver the media. Temporal offsets orother perceptible variances can arise when a media player or otherelement generating the media at one device plays or otherwise transmitsthe media after or before another, nearby device in a manner sufficientto cause the resulting signaling to arrive or be experienced at theperson at perceptively different times. The amount of time or offsetnecessary to induce the perceived variations can vary depending onreflections, wavelength and any number of other factors, such as whetherthe media has an auditory and/or a visual component.

The person may perceive temporal variations for auditory componentsdifferently than visual components, i.e., a certain amount of temporaloffset may be unperceived for auditory components and not for visualcomponents or vice-a-versa. A precedent effect, for example, has shownwhen two identical sounds are presented in close succession, such asfrom different output devices, they will be heard or perceived at theperson as a single fused sound, i.e., a sound without echo, in the eventthe corresponding signals arrive at the person with a lag or temporaloffset around 5 ms for less complex sounds, like clicks, and around 40ms for more complex sounds, like speech or piano music. An undesirableecho or other variation can be perceived at the person, such as when thetwo signals fail to provide a fused sound, in the event the lag ortemporal offset exceeds the range associated with the correspondingsound or complexity of the sound. A similar phenomenon can occur withvisual signals, such as strobes, lights, lasers, video, etc., optionallywith different tolerances for the lag, i.e., visually components mayhave shorter or longer ranges of tolerance.

One non-limiting aspect of the present invention contemplatesintelligently synchronizing media communicated from multiple outputdevices within auditory and/or visual range of a person so as tominimize and/or eliminate the perceived variations resulting fromtemporal deviations in transmission, playback and/or other influencingoperations associated with the corresponding media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a media system configured to facilitate intelligentlysynchronizing media playback in accordance with one non-limiting aspectof the present invention.

FIG. 2 illustrates a media device in accordance with one non-limitingaspect of the present invention.

FIG. 3 illustrates a street-level view of a media system in accordancewith one non-limiting aspect of the present invention.

FIG. 4 illustrates audio from multiple media devices may beintelligently synchronized in accordance with one non-limiting aspect ofthe present invention.

FIG. 5 illustrates a chart of temporal playback variations associatedwith multiple media players processing media frames in a non-linearmanner.

FIG. 6 illustrates a timing log in accordance with one non-limitingaspect of the present invention.

FIG. 7 illustrates a messaging diagram for a method of intelligentlysynchronizing media in accordance with one non-limiting aspect of thepresent invention.

DETAILED DESCRIPTION

As required, detailed embodiments of the present invention are disclosedherein; however, it is to be understood that the disclosed embodimentsare merely exemplary of the invention that may be embodied in variousand alternative forms. The figures are not necessarily to scale; somefeatures may be exaggerated or minimized to show details of particularcomponents. Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as arepresentative basis for teaching one skilled in the art to variouslyemploy the present invention.

FIG. 1 illustrates a media system 10 configured to facilitateintelligently synchronizing media playback in accordance with onenon-limiting aspect of the present invention. The system 10 may includea plurality of media devices 12 configured to facilitate communicatingor otherwise outputting media or other content, signals, etc. to asurrounding area, such as in the manner described in U.S. Pat. No.9,071,911, the disclosure of which is hereby incorporated by referencein its entirety herein. A source 14 may be remotely located to providethe media and/or suitable instructions to the media devices (such as butnot limited to a smart gateway, virtual server or cloud) 12 over anetwork 16. The network 16 may be wireline and/or wireless network(s)sufficient for facilitating communications between the source 14 and themedia devices 12, optionally in a mesh manner where the source 14communicates with one or more of the media devices 12 operable as anedge or smart gateway such that that media device 12 then subsequentlycommunicates with the remaining media devices 12, such as in the mannerdescribed in Institute of Electrical and Electronics Engineers (IEEE)Standard 820.15.4 for Local and metropolitan area networks—Part 15.4:Low-Rate Wireless Personal Area Networks (LR-WPANs), the disclosure ofwhich is hereby incorporated by reference in its entirety herein.

FIG. 2 illustrates one of the media devices 12 in accordance with onenon-limiting aspect of the present invention. The media devices 12 maybe generally characterized as any unit capable of emitting audio,visual, and/or audio-visual (video) signals to the surrounding areas.The media devices 12 may include memories, processors, communicationinterfaces and other features to facilitate the operation thereof andare shown for exemplary purposes to include an input/output 20, a player22, a memory 24, a clock 26 and a controller 28. The input/output 20 mayfacilitate exchange of control or other messages with the network and/ora user interface, display, etc. (not shown). The player 22 may cooperatethe input/output to facilitate playing media stored on the memory 24according to scheduling and other instructions of the controller 28 or aprogram associated therewith time with the clock 26. The media devices12 are shown for exemplary purposes as being deployed along a street ortrafficked throughway in order to facilitate communicating media topersons, vehicles and other objects in the vicinity thereof, however,the media devices 12 may be synchronized in the contemplated manner whendeployed in virtually any sized area.

One or more of the media devices 12 may be a lighting fixture configuredto emit light and/or other visual signals to the surrounding area. Thelighting fixtures (or any light emitting device) may be controlled toperform any number of operations, including operations associated withtheatrical lighting maneuvers, and/or controlled according to any numberof standards and protocols, including those specified in the DMX-512protocol defined by the United States Institute for Theatre Technology,Inc. (USITT). One or more of the media devices 12 may be an audio unitconfigured to emit audio signals to the surrounding area according to aplaylist of other set of instructions. Alternatively, the audio unitsmay be configured to tune to particular buffered or real-time audiostreams for broadcasting to the surrounding area. The audio unit may bea banner type speaker unit, such as that specified in United Statespatent and patent applications, 2013/0142372; 2013/0083537; U.S. Pat.Nos. 8,442,242; 8,098,852; 8,477,967; 7,219,873; 7,607,512 and6,769,509, the disclosures of which is hereby incorporated in theirentireties by reference herein. One or more of the media devices 12 maybe a video unit configured to emit video signals to the surroundingarea. The video unit may include a television screen or other displayand an audio source to facilitate emitting audio-video signals to thesurrounding area. The video units may be configured to tune toparticular buffered or real-time video streams for broadcasting to thesurrounding area and/or to receive video signals from a service provideror other entity.

The source 14 may be generally characterized as any unit (software orhardware) capable of generating instructions for controlling operationsof the media devices 12. The source 14 may include memories, processors,and other features for executing any number operations, including acommunication feature to facilitate electronic communications with themedia devices 12. The source 14 may be configured to receive and/orgenerate a control strategy for controlling operations of the mediadevices 12 and/or a communications strategy for controllingcommunications between the media devices 12. The source 14 may be astandalone feature having applications for use in controlling the mediadevices 12 and/or the source 14 itself may be an application, such asthat run by a computer or other processing means, which may be includeon one of the media devices 12, such as an edge device, for directly orindirectly controlling operation of the other media devices 12. Thesource 14 may be a software program, logic, or other feature embodied ina computer-readable medium or other suitable medium. The source 14,while shown as a feature separate from the media devices 12, may resideon one or more of the media devices 12 and need not be a separatefeature.

The source 14 may be configured to receive or store a show schedule orother feature associated with formatting multiple operations of themedia devices 12. The show schedule may include a timeline andcorresponding operations to be executed at particular intervals orevents. Queues, macros, and other features may be included within theshow schedule to facilitate changing operations and other parametersassociated with adjusting or otherwise varying operation of the mediadevices 12 to correspond with the show schedule. The control strategymay be based on the show schedule or other set of events for controllingoperations of one or more of the media devices 12. Multiple controlstrategies may be generated and distributed to the media devices 12 tocontrol the operation thereof. In particular, if the system 10 includesdifferent types of the media devices 12, multiple control strategies maybe provided for each type of media device 12. Optionally, a commoncontrol strategy may be distributed to multiple media devices 12 tocontrol the media devices 12 to cooperatively execute a number ofoperations according to a predefined schedule, such as to execute anaudio, lighting, or video show where operations of multiple mediadevices 12 are coordinated according to a common schedule.

The communications strategy may be used to control communicationsbetween the media devices 12 in accordance with IEEE 802.15.4 or othersuitable communication strategy operable to facilitate the localized,group-based broadcasts and message contemplated to enable theintelligent synchronization of the present invention. The communicationsstrategy may include features for coordinating delivery of the controlstrategy to other media devices 12. For example, the communicationsstrategy may be used to facilitate delivery of one or more controlstrategies to one or more media devices 12 so as to permit the mediadevices to be deployed in the system 10 without having the controlstrategy loaded prior to the deployment thereof and/or to facilitatedistribution of changes to the control strategy without requiring thesource 14 to directly communicate with each media device 12. Thecommunications strategy may include instructions for transportingparticular control strategies to media devices 12 associated therewith,such as to permit multiple control strategies to be transported to thesame or different media devices 12.

The communications strategy may be used to control communications ofnewly added media devices 12 with deployed media devices 12 and thesource 14. The newly added media devices 12 may be configured toregister or otherwise contact the deployed media devices 12 whenattempting to enter the system 10. The deployed media devices 12 mayconsult the communication strategy and request information from thenewly added media devices 12 to determine whether the newly added mediadevices 12 are to be added to the system 10. The other media devices 12may authenticate or otherwise restrict access to the control strategy tomedia devices 12 meeting desired security parameters. The communicationsstrategy may specify an authentication processes and other proceduresfor use in verify access to the control strategy. The approved mediadevices 12 may then be transferred the communications strategy tocoordinate communications with other media devices in the system.

Once approved for addition to the system 10, the newly added mediadevices 12 may retrieve one or more control strategies from the source14 and/or other media devices 12 according to instruction includedwithin the communications strategy. In this manner, the presentinvention is able to dynamically build an environment wherein mediadevices 12 may be freely added and controlled without requiringregistration and authentication with the source 14 or other systemadministrators. The source 14 may be configured to receive and/orgenerate a network integrity strategy. The network integrity strategymay be used to monitor the media devices 12 in the system 10 and todetermine whether the monitored media devices 12 are operation accordingto the desired control strategy. The network integrity strategy may beconfigured to periodically poll the media devices 12 and to determinewhether media devices 12 have been added or removed from the system 10.

FIG. 3 illustrates a street-level view 30 in accordance with onenon-limiting aspect of the present invention. The street-level view 30illustrates one use of the present invention to facilitate mediadelivery as well as environmental monitoring, sensing, lighting, etc.where the media devices 12 are included on a plurality of light poles32, 34, 36 illuminate a thoroughfare and/or output media thereto, suchas through playback of audio, lighting, video and the like. A polemounted digital banner 38 may be included as part of one of the mediadevices 12 to facilitate communicating video related media. A camera 40may be included as part of one of the media devices 12 to facilitatesynchronizing video or other image-based monitoring. A digital sign 42or display smaller than the digital banner 32 may be included as part ofone of the media devices 12 to indicate a street name or otherinformation. A gateway 44 may be included to facilitate wirelesscommunication with the other media devices 12, optionally as a functionof information and other data received through a wired connection. Thefollowing media devices 12 also shown for exemplary purposes: acontrollable facade light 46; a building mounted digital board 48; awater control 50 operable to facilitate controlling a water fountain; abus stop digital banner 52; an interior light controller 54 and abuilding mounted lighting fixture 56. An environmental sensor 58 and anair quality detector 60 may be included to detect triggering events forthe media devices 12, such as by collecting data and using thesynchronization from the input to synchronize an action.

One non-limiting aspect of the present invention contemplatesharmonizing, coordinating, orchestrating or otherwise synchronizingoperation of the media devices 12 in an intelligent manner, such as toenable human-perceptible media from multiple output devices to bereceived at a particular location in concert. One non-limiting aspect ofthe present invention contemplates coordinating asynchronous orsequential actions, as in strobes or other sensory messaging, where itmay be desirable for the media output devices to generate sensorymessages asynchronously or offset in time to neighboring media outputdevices, e.g., output the same strobe but at a slightly different pointin time, with the contemplated synchronization being utilized tomaintain the offset or asynchronous action. The present inventioncontemplates intelligently synchronizing virtually any type of media andis predominately described with respect to synchronizing audio forexemplary purposes as the corresponding operations and processes may besimilarly used in facilitating synchronization of other media. FIG. 4illustrates an exemplary audio schematic 70 where audio from multiplemedia devices 72, 74, 76 may be intelligently synchronized in accordancewith one non-limiting aspect of the present invention. The illustratedmedia devices 72, 74, 76 may include a luminaire for illuminating anambient environment associated therewith, optionally according toschedule or controllable methodology, and an audio player, decoder orother device operable to interface sound waves with the environment. Theaudio is shown as a plurality of audio frames (shown as musical symbols)being played from multiple media devices 72, 74, 76 to a person withinan auditory range thereof. The audio frames may be stored on the mediadevices 72, 74, 76 within a file and/or a playlist, e.g., a song orwarning message may be included in a single file and multiple files maybe arranged for playback according to a playlist.

The audio frames and/or audio players may be formatted or operable toprocess Moving Pictures Expert Group (MPEG) frames, such as thosecompliant with International Organization for Standardization (ISO)standard ISO/IEC 11172-3 (MPEG-1 Audio), which defines three formats:the MPEG-1 Audio Layer I, Layer II and Layer III; and the ISO standardISO/IEC 13818-3 (MPEG-2 Audio), which defines extended version of theMPEG-1 Audio: MPEG-2 Audio Layer I, Layer II and Layer III (MP3), thedisclosures of which are hereby incorporated by reference in theirentireties herein. Whether defined according to one of the MPEGstandards or other standards, the audio frames may each include a header(not shown) and a payload (not shown) with the header includingidentifying information and other data associated with organizing,communicating or otherwise facilitating use of the audio frames and thepayloads may include the audio samples to be played, i.e., the music,sounds, warnings or other perceptible auditory components. The audioframes are predominately described as one type of media haveindividually definable and identifiable segments requiring sequential orchronological playback in order to create a desired sound(s) (or alighting event for path of egress) and is presented for exemplarynon-limiting purposes as other audio formats besides frames may besimilarly used without deviating from the scope and contemplation of thepresent invention.

The audio frames may be arranged in a chronological order whereby theaudio frames are played in a particular, numbered sequence to deliverthe desired sounds or series of sounds, such as by playing frame Nfollowed by frame N+1, then frame N+2 . . . frame N+n where n is theending frame of the song, message, sound, etc. One non-limiting aspectof the present invention contemplates utilizing the frame ordering tofacilitate intelligently synchronizing audio for multiple media outputdevices 72, 74, 76 so as to prevent echoes and other undesirabletemporal variations. The contemplated synchronization may includeselecting one of the media devices, such as the first media device 72,to act as a conductor for the remaining media devices 74, 76, which maybe referred to as non-conductors (any number of conductors andnon-conductors may be employed). The conductor 72 may be configured totransmit sync messages to the non-conductors 74, 76identifying when ithas or will play a particular one of the audio frames. The conductor 72may identify epochs or other temporal references sufficient to associateone or more of the audio frames with a determinable period in time. Thenon-conductors 74, 76may compare their playback variables, such asspeed, location, encoding, bit rate, etc., to the conductor epochsand/or other information included in the sync messages in order toindividually time playback thereto.

The sync messages may each associate an epoch with a frame identifierand/or other playback related parameters that the non-conductors 74, 76can then use to adjust their operation to be in concert with theconductor 72. The sync messages may be particularly beneficial inallowing the non-conductors 74, 76 to individual adjust for playbackvariations of the media players, such by pausing, skipping, fastforwarding or implementing other trick modes sufficient to time playbackrelative to the conductor. FIG. 5 illustrates a chart 80 of temporalplayback variations associated with multiple media players 72, 74, 76processing media frames (e.g., audio, video or other content frames) ina non-linear manner. The chart 80 graphs a speed or playback rate of themedia players/media devices 72, 74, 76 varying slightly over time from abeginning frame (left-hand side) to an ending frame (right-hand side).The playback rates may be considered as non-linear due to the mediaplayers 72, 74, 76 individually processing the media frames for outputat different rates as a result slight variations in the attendantdecoding and/or output related operations. The media players 72, 74, 76may experience temporal variations in an unpredictable or non-linearmanner each time media is played such that the curve or graph of eachmedia player 72, 74, 76 may vary over time and/or vary as a function ofthe media being played, i.e., less complex sounds may have one curve andmore complex sounds may have a different curve.

The variable playback rates can influence the sensory perception of a 82person within the corresponding auditory range (shaded area of two ormore media devices 72, 74, 76) should the resulting variations cause therelated signaling to arrive at perceptibly different intervals. Thetemporal range or offset sufficient to induce perceptible variations mayvary depending on the complexity of the sound and other signalingparameters (e.g., reflection, propagation, attenuation, etc.). Thecontroller 28 of each media device 72, 74, 76 may include an applicationcapable of processing the information included in the sync messages tofacilitate adjusting the playback rate (e.g., shape of the correspondingcurve) of the corresponding media player 72, 74, 76 by slowing orincreasing processing rates to at least approximate the playback rate ofthe conductor 72 in a manner sufficient to prevent echoes or otherundesirable temporal variations. Given the unknown or non-linearperformance of each media player 72, 74, 76, it may be difficult toadjust the playback rates or processing speeds of the media players tocoincide with the conductor 72 such that other measures may be requiredto properly time playback of the audio frames.

One non-limiting aspect of the present invention contemplates theconductor 72 transmitting a plurality of sync messages, optionally on anintermittent or periodic basis, to enable the non-conductors 74, 76 tocontinuously adjust their playback position to coincide with theconductor 72. The playback positioning adjustments may correspond withthe corresponding controller 28 identifying whether the correspondingmedia player 74, 76 is in sync or ahead/behind the conductor 72 i.e.,whether the media device 74, 76 is playing or has played a particularframe at the same time that the conducted 72 has played or is playingthe corresponding frame or whether the media device 74, 76 is playing orhas played a particular frame before/after the conductor 72 has playedor is playing the corresponding frame. The corresponding playbackadjustments may include but are not limited to pausing playback in theevent the media device 74, 76 is ahead of the conductor 72, skipping orforwarding playback in the event the media device 74, 76 is behind theconductor 72 or maintaining current operation in the event the mediadevice 74, 76 is in sync with the conductor 72.

The sync messages may be associated with the conductor 72 playing orotherwise processing one of the audio frames for output, i.e., uponbeginning to process the audio frame to determine the correspondingaudio signaling and/or upon actually outputting the corresponding audiosignal. The lag or delay associated with transmitting the sync messagesmay require the corresponding controllers 28 of the non-conductors 74,76 to look backward for purposes of comparing their play history to theconductor 72 and thereafter adjust future playback accordingly. FIG. 6illustrates a timing log 86 useful as one contemplated tool tofacilitate keeping historical playback records in accordance with onenon-limiting aspect of the present invention. The timing log 86 may bekept by each of the controllers 28 to identify when one or more of theaudio frames associated with the audio file, playlist or sequence wasplayed at the corresponding media player 22. While the controllers 28may log each audio frame associated with the file or multiple files,FIG. 6 illustrates an exemplary scenario where the timing log 86 keeps arecord of when the audio frames associated with the sync messages wereoutput (referred to as sync frames). The sync frames may be determinedas a function of information included in the corresponding frame headerand/or based on positioning from a first frame in the chronologicalorder, e.g., every x-number of frames from the beginning frame may beconsidered as a sync frame for tracking purposes.

The timing log 86 may include a first column 88 associated withidentifying the sync frames, a second column 90 associated withidentifying when the media player 22 of the corresponding non-conductor74 output the matching sync frame, a third column 92 associated withwhen the media player 22 of the conductor 72 output the same sync frameand a fourth column 94 indicating a playback adjustment, if any, neededto re-adjust playback positioning of the corresponding non-conductor 74.The non-conductor and conductor timing may be set relative to epochs orother independent temporal references, which may optionally be set tothe clock of the associated media device 72, 74, 76. The media devices72, 74, 76 or the controller 28 associated therewith may includecapabilities sufficient to execute network time protocol (NTP), such asthrough communications within the mesh network 16 and/or the source 14,or other timing operations sufficient to coordinate the clock 26 of eachmedia device 72, 74, 76 to a common reference within an acceptable rangeof tolerance. One non-limiting aspect of the present inventioncontemplates assuming the clocks 26 of each media device 72, 74, 76 aresufficiently aligned such that the epochs indicated in the timing log 86are a sufficient approximation of when each media player 22 output orprocessed the corresponding media frame for output.

The media device controllers 28 may keep track of the epochs listed inthe second column 90 and then compare those epochs to the conductorepochs provided in the sync messages as shown in the third column 92.Corrective action may be implemented thereafter to facilitatecontrolling the frame positioning of the corresponding media player 22to match playback of future frames to the conductor 72. FIG. 6illustrates an exemplary scenario where the non-conductor 74 steps offor begins playing a first sync frame (S) in concert with the conductor72, such as in response to a step-off message from the conductor 72indicating when each non-conductor 74 is to begin. The log 86 indicatesthe non-conductor 74 thereafter beginning to slow until a sufficientoffset occurs at a fifth sync frame (S4) whereupon an adjustment is madeto forward the media player 22. The forwarding may include skipping oneor more of the intervening frames between successive sync messages (anumber of frames, referred to as non-sync frames, may be defined andspecified in the chronological order or playlist between two syncframes) and/or the sync frames themselves may be skipped such that themedia player 22 outputs a sixth sync frame (S5) and/or frames precedingthe sixth sync frame (S5) at the same time as the conductor 72.

As shown, the non-conductor 74 slowly falls out of sync with theconductor 72 by a third sync frame (S3) with corrective action occurringlater once the offset exceeds an acceptable threshold. The acceptablethreshold may be set for each media device 72, 74, 76 according toinstructions from the conductor 72 and/or the source 14 and mayoptionally be varied depending on the audio file or content type and/orsignaling transmission characteristics (reflection, attenuation, etc.).The amount of forwarding may correspond with the acceptable threshold oran actual amount by which the non-conductor 74 is behind the conductor,e.g., a preset number of frames may be skipped equaling the thresholdand/or an elapse time between corresponding conductor and non-conductorepochs may be used to approximate a duration of the forwarding. The timeor other marker of duration may be related to fame identifiersassociated with each audio frame header so as time playback of the nextor subsequent frames to be in concert with the conductor 72. Thenon-conductor 74 is shown to keep pace with the conductor 72 after thepausing for a period of time corresponding with a seventh and eighthsync frame (S6, S7) before moving ahead of the conductor by an amountexceeding the acceptable threshold at an eleventh sync frame (S10)whereafter corrective action is taken to pause the media player 22.

The pausing may include preventing the media player 22 from playing oneor more of the audio frames until playback of the audio frames occurringafter the eleventh sync frame (S10) can play in concert with theconductor 72. A duration of the pausing may be determined similarly tothe forwarding insofar as being set to a preset number of frames orlength of time corresponding with the acceptable threshold and/or anactual elapse or difference between the non-conductor and conductorepochs. As operation of the non-conductor 74 may be non-linear, thepausing, forwarding or other corrective action based on actual elapsetime differences between conductor and non-conductor epochs may bepreferred over preset or fixed adjustments, particularly whencompensating for rapid or extreme variations as use of presetadjustments may not be sufficient to align the non-conductor 74 with theconductor 72. The use of pausing and/or forwarding as corrective actionsare described merely as exemplary processes for intelligentlysynchronizing the non-conductors 74, 76 to the conductor 72 as thepresent invention fully contemplates other manipulations, such asplaying the audio files at a faster/slower rate, e.g., seeking orplaying the audio frames at 1.5×, 2× or other rate values faster/slowerthan the conductor in proportion to the corresponding timingdifferences.

The timing log 86 is also presented for exemplary purposes todemonstrate certain values and information kept at the media devices 72,74, 76 to facilitate the intelligent synchronization contemplated by thepresent invention. The controllers 28 may include algorithms or othermechanisms for tracking playback and facilitating alignment withoutnecessary logging the sync and non-sync frames. The timing log 86 isillustrated as tracking only the sync frames for exemplary purposes asthe non-sync frames between the sync frames may be similarly tracked forcomparison purposes, such as to enable the controller 28 to compare orestimate when the conductor 72 has or will output the non-sync frames,i.e., timing of the conductor 72 for non-sync frames or frames for whichsync messages were not transferred may be interpolated from the syncframe data. The lag or delay associated with receipt and processing ofthe sync messages, including relating the sync messages to informationtabulated in the timing log 86, may result in the correspondingcontrollers 28 looking backward, i.e., back in time to assess playbackof frames occurring before receipt of sync message, in order to assesstemporal difference relative to the conductor 72 for purposes of usingthose differences to adjust future playback in the contemplated manner.

FIG. 7 illustrates a messaging diagram 100 for a method of intelligentlysynchronizing media in accordance with one non-limiting aspect of thepresent invention. The method may be embodied in a computer-readablemedium having a plurality of non-transitory instructions operable with aprocessor associated with one of the above-described media devices 12,source 14 or other entity having capabilities sufficient to facilitateoperations contemplated herein. The above description predominatelydescribed operation of the present invention with respect tofacilitating synchronization of an audio file transmitted as a pluralityof audio frames from a plurality of media devices 12 within an auditoryrange of each other for exemplary non-limiting purposes. The method ispredominately described with respect to facilitating intelligentsynchronization for a media file or multiple immediate files, such asthose specified within a playlist, schedule or other schema. The mediafile may be divided into a plurality of media frames and be synchronizedsimilarly to the audio file and frames described above and is referencedto further highlight the capability of the present invention tofacilitate synchronizing virtually any type of content, such as but notnecessary limited to video, lighting, audio and other sensory-based orperceptible messaging, signaling or material (e.g., water of a waterfountain). The contemplated synchronization is predominately describedwith respect to coordinating the media devices to output the samemessage at approximately the same time for exemplary purposes as thecontemplated synchronization may similarly coordinate multiple mediaoutput devices to output the same or different media messages (or mediafiles) in an asynchronous/sequential manner.

The use of frames to designate segments or divisions of a larger fileuseful in facilitating the contemplated intelligent synchronization isalso presented for exemplary non-limiting purposes as other demarcationsor suitable points within the media file may be similar employed withoutdeviating from the scope and contemplation of the present invention. Thepresent invention may be particularly beneficial in facilitymulti-source synchronization of streamed media or other forms of mediadistributed using markers other than frames to facilitate arranging therelated data for playback, e.g., the envisioned synchronization may besimilarly achieved through packet-based demarcations, timing references,etc. One non-limiting aspect the present invention presumes the mediafile or media frames being pre-loaded or otherwise stored at the mediadevices 12 prior to beginning playback, thereby eliminating the need forthe synchronization processes to compensate for media deliveryinconsistencies during playback. This is done merely to facilitatedescribing the fundamentals of the present invention as mediatransmission delays may be considered in adjusting operation ofnon-conductors without deviating from the scope and contemplation of thepresent invention, such by adjusting the length of the pausing/forwardaccording to corresponding transmission influences.

FIG. 7 illustrates an exemplary environment where five media devices102, 104, 106, 108, 110 are positioned on movable or immovable objectswithin a geographical area resulting in each of the media devices 102,104, 106, 108, 110 being within an auditory/acoustical and/or visualrange of least one of the other media devices 102, 104, 106, 108, 110.The geographical area may include one or more subjected regions or zonesreachable with signaling from two or more of the media devices 102, 104,106, 108, 110, which may be referred to as acoustical areas whenreachable with audio signals and a visual area when reachable withvisual signals. One non-limiting aspect of the present inventioncontemplates determining the subjected areas and other informationassociated with controlling sensory message (media) synchronization fromnon-sensory messages exchanged between the media devices 102, 104, 106,108, 110. The influenced areas may be determined as a function ofinformation, capabilities, data, etc. gleaned from a first plurality ofping messages 114, 116, 118 exchanged between the media devices 102,104, 106, 108 according to communication capabilities of a correspondingmesh network or other network configuration associated therewith.

The first plurality of ping messages 114, 116, 118 may be utilized toglean location, capabilities, signaling, media and other information forthe media devices 102, 104, 106, 108 associated therewith. A conductorprocess (not shown) may be implemented as a function of one or more thefirst plurality of ping messages 114, 116, 118 whereupon one of themedia devices 102 may be determined to a conductor and the remainingmedia devices 104, 106, 108 determined to be non-conductors. Optionally,the conductor process may be similar to a coordinator process describedin IEEE 802.15.4 for identifying a coordinator and/or may includedetermining one of the non-conductors 104 to be a backup conductor inthe event conductor fails or is unable to broadcast sync message for apredetermined period of time (e.g., 30 ms or other interval associatedwith media playback rates). One of the media devices 110, which may besubsequently included as described below, may be initially beingexcluded from the first plurality of ping messages 114, 116, 118 due tobeing unavailable or unassociated with the networking of the other mediadevices 102, 104, 106, 108 when the first plurality of ping messages114, 116, 118 were exchanged.

A latency process 120, 122, 124, 126 may occur at the conductor 102and/or the non-conductors 104, 106, 108 to determine an amount oflatency associated with signaling traveling therebetween. The latencyprocesses 120, 122, 124, 126 may include determining travel timeindividually for messaging between the conductor 102 and eachnon-conductor 104, 106, 108 and/or between each of the non-conductors104, 106, 108. The conductor 102 may thereafter transmit a step-offmessage 130 or multiple step-off messages 132, 134 to schedule each ofthe non-conductors 104, 106, 108 to begin playing the media file at anepoch or other discernible interval, such as in response to one or moreof the media devices 104, 106, 108 individually sensing an environmentalcondition or a localized event unknown to the conductor 102. Assuminguse of a step-off epoch, the timing may be selected as a function of thelatency processing to schedule playback at a point in time sufficientfor the step-off messages 130, 132, 134 to reach each media device 104,106, 108 before playback is planned to commence. The step-off messages130, 132, 134 may be communicated directly from the conductor 102 to thenon-conductors 104, 106, 108 and/or relayed from one non-conductor 104,106, 108 to one non-conductor 104, 106, 108 as with the ping messages114, 116, 118. The step-off messages 130, 132, 134, in addition toidentifying the step-off time, may identify the media file and/ormultiple media files to be played thereafter. One non-limiting aspect ofthe present invention contemplates transmitting additional step-offmessages (not shown), optionally at different times, so as to provide athink ahead process for preparing the media devices 102, 104, 106, 108to output subsequent media files before the currently playing media fileceases.

A playback process 136, 138, 140, 142 may occur at each of the mediadevices 102, 104, 106, 108 upon detecting occurrence of a triggeringevent associated with the step-off messages 130, 132, 134, i.e., uponthe corresponding clock determining occurrence of the step-off time. Theplayback processes 136, 138, 140, 142 may include each media device 102,104, 106, 108 autonomously beginning and continuing to play the mediafile without adjustment until receiving a sync message or otherinstructions from the conductor 102. The playback processes 136, 138,140, 142 may include the media devices 102, 104, 106, 108 performingoperations associated with communicating the media frames or othersegments of the media file to an ambient environment associatedtherewith, which may include the media devices 102, 104, 106, 108generating acoustical and/or visual signaling for acoustical and/orvisual components included within the media payload or otherwiseproviding sensory or perceptible messages, content, material, etc. tothe surrounding area. The playback processes 136, 138, 140, 142 may alsoinclude the media devices 102, 104, 106, 108 generating a timing log orotherwise keeping track of the media frames during playback so as tofacilitate the calculations associated with implementing adjustmentsnecessary to keep synchronized with the conductor 102, optionally tofacilitate the pausing and forwarding operations described above.

A conductor process 144 may correspond with the conductor 102transmitting one or more sync messages 148, 150, 152 associated with theconductor 102 playing one or more of the media frames. The sync messages148, 150, 152 may be transmitted from the conductor 102 to each of thenon-conductors 104, 106, 108 individually and/or relayed from onenon-conductor 104, 106, 108 to another non-conductor 104, 106, 108. Thenon-conductors 104, 106, 108 may execute a plurality of adjustments 156,158, 160 or other operations as a function of information includedwithin the sync messages 148, 150, 152 to facilitate coordinatingmessage output in concert with the conductor 102. The conductor 102 maythereafter continue conducting 164 and transmitting additional syncmessage 166, 168, 170 and the non-conductors 104, 106, 108 may continueto make any necessary adjustments 172, 174, 176 as a function thereof.The sync messages 166, 168, 170 may optionally be tailored to thearea(s) reachable with at least two of the media devices 104, 106, 108such that different sync messages 166, 168, 170 may be transmitted todifferent media devices 104, 106, 108 depending on the requiredcoordination. This may include the conductor 102 conducting groups oftwo or more non-conductors 104, 106, 108 differently and/or otherinstructing the grouping to make adjustments differently depending oninformation included the sync messages 166, 168, 170, which may bebeneficial in adjusting media output as a person or object moves betweenthe media devices 104, 106, 108.

Assuming each of the non-conductors 104, 106, 108 are intended tosynchronize media output to the same area, each non-conductor 104, 106,108 may be configured to facilitate adjusting 156, 158, 160 itsoperation to match the conductor 102. This may include, for example, afirst non-conductor 104 continuing its current playback rate, a secondnon-conductor 106 pausing playback and/or a third non-conductor 108forwarding playback. Advantageously, the intelligent synchronization mayoccur without the conductor 102 having to sample the media files whenoutput, such as to avoid the need for generating and storing recordingof audio or video sampling in order to facilitate synchronizing thenon-conductors 104, 106, 108. A second plurality of ping messages 178,180, 182, 184 may be exchanged between the conductor 102 non-conductors104, 106, 108, 110 in the event a new media device 110 becomes availableor otherwise attempts to act in concert with the currently active mediadevices 102, 104, 106, 108. The second plurality of ping messages 178,180, 182, 184 may be similar to the first plurality of ping messages114, 116, 118 and sufficient to facilitate determining whether theconductor 102 is to remain as a conductor and/or whether the new mediadevice 110 should become the conductor and/or to facilitate identifyingcapabilities and other information for the new media device 110.

The conductor may continue conducting 186 and thereafter transmitsubsequent sync messages 188, 190, 192, 194 to the previousnon-conductors 104, 106, 108 as well as the new non-conductor 110. Thenon-conductors 104, 106, 108, 110 may thereafter facilitate adjustingoperation 198, 200, 202, 204, if necessary, of the corresponding mediaplayers to match the conductor 102. The new non-conductor 110 mayconsider the corresponding sync message 194 as a step-off message or tootherwise instigate playback to coincide with a suitable video frame ofthe conductor 102. A backup conductor process 206 is shown to occur uponthe conductor 102 failing to continue transmitting sync messages or upona need for the conductor 102 to transmit the media expiring, e.g., theconductor 102 may be selected for a brief period of time while a user iswithin the vicinity and thereafter become unnecessary once the usermoves and/or the conductor 102 may be selected to conduct while sensingor otherwise determining the triggering event and thereafter ceaseconducting when the triggering event ends. The previously determinedbackup conductor 104 may then instigate a conductor process 206 similarto the above-described conductor process whereupon sync messages 208,210, 212 may be communicated to the other or remaining non-conductors106, 108, 110 to facilitate syncing intelligently syncing 218, 220, 222the operation thereof to the backup conductor.

While exemplary embodiments are described above, it is not intended thatthese embodiments describe all possible forms of the invention. Rather,the words used in the specification are words of description rather thanlimitation, and it is understood that various changes may be madewithout departing from the spirit and scope of the invention.Additionally, the features of various implementing embodiments may becombined to form further embodiments of the invention.

What is claimed is:
 1. A method for intelligently synchronizing playbackof a media file from a plurality of media players to compensate forplayback variations associated with the plurality of media playersattempting to provide in concert playback of a plurality of media framesincluded in the media file, the method comprising: designating one ofthe plurality of media players to be a conductor and the remaining oneor more of the plurality media players to be non-conductors;intermittently broadcasting a plurality of sync messages from theconductor to the non-conductors as the conductor plays the media file,each of the plurality of sync messages identifying a conductor epochassociated with the conductor playing a corresponding one of theplurality of media frames; and synchronizing playback of thenon-conductors to compensate for playback variations using the pluralityof sync messages to facilitate determining temporal differences betweenwhen the conductor and the non-conductors play the media framesassociated with the conductor epochs.
 2. The method of claim 1 furthercomprising generating individual timing logs for each of thenon-conductors, the timing logs identifying non-conductor epochsassociated with the corresponding non-conductor playing a correspondingone of the plurality of media frames.
 3. The method of claim 2 furthercomprising synchronizing playback of a first non-conductor of thenon-conductors by pausing play of the media file for a first period oftime approximately equal to time elapsing between one of thenon-conductor epochs of the first non-conductor and a corresponding oneof the conductor epochs.
 4. The method of claim 3 further comprisingsynchronizing playback of a second non-conductor of the non-conductorsby forwarding play of the media file for a second period of timeapproximately equal to time elapsing between one of the non-conductorepochs of the second non-conductor and a corresponding one of theconductor epochs.
 5. The method of claim 4 further comprising pausingplay of the first non-conductor while at a same time forwarding play ofthe second non-conductor.
 6. The method of claim 1 further comprising:designating one of the non-conductors to be a backup conductor; and uponfailure of the conductor to continue broadcasting one or more of theplurality of sync messages: i) intermittently broadcasting a pluralityof backup sync messages from the backup conductor, each of the pluralityof backup sync messages identifying a backup epoch associated with thebackup conductor playing a corresponding one of the plurality of mediaframes; and ii) synchronizing playback of the non-conductors tocompensate for playback variations using the plurality of backup syncmessages to facilitate determining temporal differences between when thebackup conductor and the non-conductors play the media frames associatedwith the backup conductor epochs.
 7. A system for intelligentlysynchronizing sensory messages of the type having an auditory componentand/or a visual component, the system comprising: a first devicepositioned at a first location having: i) a first player configured toplayback a plurality of sensory messages to a first ambient environmentaccording to an order specified in a playlist; ii) a first controllerconfigured to periodically broadcast non-sensory sync messages duringplayback of the playlist, each sync message specifying an epochassociated with when the first player played a corresponding one of theplurality of sensory messages; a second device positioned at a secondlocation sufficiently close to the first location to be within anauditory range and/or a visual range of the first device, the seconddevice having: i) a second player configured to playback the pluralityof sensory messages to a second ambient environment according to theorder specified in the playlist; and ii) a second controller configuredto adjust for temporal variations while the second player plays theplaylist as a function of the epochs specified in the sync messages soas to intelligently synchronize the second player to the first player.8. The system of claim 7 wherein the second controller is configured topause the second player when one or more of the epochs indicate thesecond player to be ahead of the first player.
 9. The system of claim 8wherein the second controller is configured to: generate a timing logidentifying a time when the secondary player plays one or more of thesensory messages; and determine a length of the pause to approximatelyequal an amount of time elapsing between at least one of the times andthe epoch identified in the sync message for the corresponding one ofthe sensory messages.
 10. The system of claim 7 wherein the secondcontroller is configured to forward the second player when one or moreof the epochs indicate the second player to be behind of the firstplayer.
 11. The system of claim 7 further comprising a third devicepositioned at a third location sufficiently close to at least one of thefirst location and the second location to be within an auditory rangeand/or a visual range of at least one of the first device and the seconddevice, the third device having: a third player configured to playbackthe plurality of sensory messages to a third ambient environmentaccording to the order specified in the playlist; and a third controllerconfigured to: i) adjust for temporal variations while the third playerplays the playlist as a function of the epochs specified in the syncmessages so as to intelligently synchronize the third player to thefirst player while the first controller is broadcasting the plurality ofsync messages; and ii) periodically broadcast non-sensory, backup syncmessages during playback of the playlist when the first controller failsto broadcast the plurality of sync messages, each backup sync messagespecifying a backup epoch associated with when the third player plays acorresponding one of the plurality of sensory messages.
 12. The systemof claim 11 wherein: the first controller is configured to broadcast anon-sensory, step-off message to the second and third controllerssufficient to facilitate starting playback of the playlist at the secondand third player in concert with playback of the playlist at the firstplayer; and the second controller is configured to adjust for temporalvariations while the second player plays the playlist as a function ofthe backup epochs specified in the backup sync messages so as tointelligently synchronize the second player to the third player when thefirst controller fails to broadcast the plurality of sync messages. 13.The system of claim 12 further comprising a source configured towireless transmit the plurality of sensory messages to the first, secondand third devices for playback prior to the first, second and thirdplayers starting playback of the playlist.
 14. A method forintelligently synchronizing playback of a media file having a pluralityof media frames arranged in a chronological order, the methodcomprising: determining a plurality of media players forming anacoustical area characterized by each of the plurality of media playersbeing within an auditory range of at least another one of the pluralityof media players; determining a first epoch associated with theplurality of media players beginning concerted playback of the mediafile starting with a first media frame of the plurality of media frames;determining a second epoch associated with a first media player of theplurality of players playing a second media frame of the plurality ofmedia frames occurring in the chronological order after the first mediaframe; and synchronizing at least a second media player and a thirdmedia player of the plurality of media players to playback a third mediaframe of the plurality of media frames occurring in the chronologicalorder after the second media frame relative to the second epoch so as tointelligently synchronize at least the second and third media playersrelative to the first media player.
 15. The method of claim 14 furthercomprising facilitating pausing playback of the media file at the secondmedia player for a first length of time whereafter the second mediaplayer then begins playback of the media file with the third media frameplaying at a third epoch, the third epoch coinciding with the firstmedia player playing the third media frame.
 16. The method of claim 13further comprising pausing playback of the second media player duringthe first length at a fourth media frame of the plurality of mediaframes occurring immediately before the third media frame in thechronological order.
 17. The method of claim 14 further comprisingfacilitating forwarding playback of the media file at the third mediaplayer for a second length of time whereafter the third media playerthen begins playback of the media file with the third media frame at athird epoch, the third epoch coinciding with the first media playerplaying the third media frame.
 18. The method of claim 17 furthercomprising forwarding playback of the second media player during thesecond length such that at least a fifth media frame of the plurality ofmedia frames occurring before the third media frame in the chronologicalorder is skipped.
 19. The method of claim 14 further comprisingtransmitting a frame sync message identifying the second epoch and thesecond media frame to facilitate synchronizing at least the second andthird media players, the frame sync message being operable withsynchronization controllers operating at the second and third mediaplayers to adjust playback of the media file therefrom.
 20. The methodclaim 14 further comprising transmitting an entirety of the media fileto each of the plurality of media players in advance of the first epoch.21. A method for intelligently synchronizing playback of a plurality ofmedia players to compensate for playback variations, the methodcomprising: designating one of the plurality of media players to be aconductor and the remaining one or more of the plurality media playersto be non-conductors; intermittently broadcasting a plurality of syncmessages from the conductor to the non-conductors as the conductor playsa first plurality of media signals, each of the plurality of syncmessages identifying a conductor epoch associated with the conductorplaying a corresponding one of the first of plurality of media signals;and individually synchronizing playback of the non-conductors receivingone or more of the plurality of sync messages, including adjustingplayback of each non-conductors as a function of temporal differencesbetween when the corresponding non-conductor played a second pluralityof media signals related to the media signals associated with theconductor epochs identified in the one or more of the plurality of syncmessages received at the corresponding non-conductor.
 22. The methodclaim 21 further comprising selecting the first and second plurality ofmedia signals such that at least a portion of the second plurality ofmedia signals are different from the first plurality of media signals.23. The method of claim 21 further comprising selecting the first andsecond plurality of media signals such that the first and secondplurality of media signals are the same.
 24. The method claim 23 furthercomprising providing a playlist to each of the conductor andnon-conductors specifying an order for playback of the first and secondplurality of media signals, including arranging the playlist for one ormore of the non-conductors with the order being different than the orderin the playlist for the conductor.